Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make create_virtual_dataset handle zero-length raw_data correctly #315

Merged

Conversation

peytondmurray
Copy link
Collaborator

@peytondmurray peytondmurray commented Mar 13, 2024

Summary

This PR makes create_virtual_dataset correctly handle the case where the raw data is empty, returning an empty virtual dataset.

Closes #314.

Changes

  • If the raw data for a dataset is empty, the virtual dataset is also empty instead of being shape (1,) (??)
  • Added tests of both the backend (to see if zero-length raw datasets are correctly handled) and of the API (to cover the reproducer submitted with the original issue)
  • Added ANN001 (missing type annotations) to the list of rules to ignore in for test functions. These are either parametrizations or pytest fixtures.

I'm still not entirely sure why it is necessary to close/open the file descriptor between each operation to reproduce this issue, but it must be something to do with flushing writes to the file. @ArvidJB any ideas?

@peytondmurray peytondmurray requested a review from ArvidJB March 13, 2024 17:19
@peytondmurray
Copy link
Collaborator Author

Merging this as is to close out this bug. If any suspicious behavior persists, we can open a new issue.

@peytondmurray peytondmurray merged commit a0d1f08 into deshaw:master Mar 25, 2024
7 checks passed
@peytondmurray peytondmurray deleted the create-empty-dataset-fix-314 branch March 25, 2024 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug when creating empty dataset - PyInf#12125
1 participant